feat: support mount SMB Azure File with user-provided OAuth token through k8s secret by andyzhangx · Pull Request #3100 · kubernetes-sigs/azurefile-csi-driver

andyzhangx · 2026-04-21T11:50:45Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

Add mountWithOAuthToken volume context parameter to support cross-tenant scenarios where users provide their own OAuth token via a Kubernetes Secret.

This enables first-party users who cannot use kubelet identity or workload identity (cross-tenant) to mount Azure File shares using an externally-managed OAuth token.

Changes:

New volume context parameter: mountWithOAuthToken
New secret field: oauthtoken
Extend setCredentialCache to support direct token passing to azfilesauthmanager set
Extend GetStorageAccountFromSecret to return OAuth token from secret
New method setCredentialCacheWithOAuthToken — reads OAuth token from K8s Secret and updates credential cache
Mutually exclusive with mountWithManagedIdentity and mountWithWorkloadIdentityToken
Linux SMB only (sec=krb5 mount)
Supports both dynamic provisioning (via StorageClass parameters) and static provisioning (via PV volumeAttributes)
NodeStageVolume: performs initial mount with OAuth token credential cache setup
NodePublishVolume: refreshes OAuth token credential cache on every publish (even when already mounted), with SHA256-based skip to avoid unnecessary azfilesauthmanager calls when the token is unchanged
getSecretNamespace helper for consistent namespace resolution (secretNamespace → pvcNamespace → "default")
Input validation: secretName required, empty server returns InvalidArgument

Usage Example

1. Create Secret with OAuth Token

kubectl create secret generic azure-oauth-token-secret --from-literal=oauthtoken="<oauth-token>"

2. Dynamic Provisioning (StorageClass)

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: azurefile-oauth-token-sc
provisioner: file.csi.azure.com
parameters:
  skuName: Premium_LRS
  mountWithOAuthToken: "true"
  secretName: "azure-oauth-token-secret"
  secretNamespace: "default"
mountOptions:
  - sec=krb5

3. PV (Static Provisioning)

apiVersion: v1
kind: PersistentVolume
metadata:
  name: azurefile-oauth-token-pv
spec:
  capacity:
    storage: 100Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  csi:
    driver: file.csi.azure.com
    volumeHandle: "unique-volume-id"
    volumeAttributes:
      storageAccount: "mystorageaccount"
      shareName: "myshare"
      mountWithOAuthToken: "true"
      secretName: "azure-oauth-token-secret"
      secretNamespace: "default"

Security Considerations

Important: The OAuth token secret should be treated as cluster-scoped credentials. The secret reference comes from the PV/StorageClass (provisioned by a cluster admin), but any pod that can mount the PV gets the secret's privileges. There is no per-pod scoping.

Best practices:

Store the OAuth token secret in a namespace that is not writable by tenants (e.g., kube-system or a dedicated admin namespace)

Use secretNamespace to explicitly point to the admin-controlled namespace

Restrict RBAC so that workload namespaces cannot read or modify the token secret

Which issue(s) this PR fixes:

Fixes #3099

Does this PR introduce a user-facing change?

Support mounting Azure File shares with user-provided OAuth token via `mountWithOAuthToken` volume context parameter for cross-tenant scenarios. Supports both dynamic provisioning (StorageClass) and static provisioning (PV). Token credential cache is refreshed on NodePublishVolume with SHA256-based deduplication.

if token is expired, you would hit following error when read/write data from azure file mount:

│ W0427 03:24:50.439219 1016124 nodeserver.go:841] ReadDir /var/lib/kubelet/pods/90ae2394-8dc4-4f23-ad79-2f4ae3c47d47/volumes/kubernetes.io~csi/pv-azurefile/mount failed with readdirent /var/lib/kubelet/pods/90ae2394-8dc4-4f23-ad79-2f4ae3c47d47/volumes/kubernetes.io~csi/pv-azur │
│ efile/mount: required key not available, unmount this directory

Copilot

Pull request overview

Adds support for mounting Azure File shares using a user-provided managed identity OAuth token (via Kubernetes Secret) to enable cross-tenant scenarios, integrating this as a new volume context parameter and wiring it into NodeStageVolume credential-cache setup.

Changes:

Add mountwithmanagedidentitytoken volume context parameter and secret field azurestoragemanagedidentitytoken.
Extend credential-cache setup to support passing an OAuth token directly to azfilesauthmanager set.
Update node staging logic to handle the new MI-token path and enforce mutual exclusivity with existing identity-based options.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
pkg/azurefile/utils.go	Extends `setCredentialCache` to accept either tokenFile-based WI flow or direct token flow; redacts token in logs.
pkg/azurefile/utils_test.go	Updates `setCredentialCache` call signature in tests.
pkg/azurefile/nodeserver.go	Adds parsing/validation for new volume context flag and implements `setCredentialCacheWithMIToken` called during SMB mount.
pkg/azurefile/nodeserver_test.go	Updates mutual exclusivity test/error message to include the new flag.
pkg/azurefile/azurefile.go	Adds new constants and extends `GetStorageAccountFromSecret` to return the MI token; recognizes the new mount mode in `GetAccountInfo`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Use status.Errorf instead of fmt.Errorf so the error surfaces as codes.Internal rather than codes.Unknown through status.Convert.

- CreateVolume: validate secretName is provided when mountWithOAuthToken is true, fail early with InvalidArgument instead of deferring failure to NodePublish - e2e: use shared oauthSecretName/oauthSecretNamespace constants instead of hardcoded strings in dynamic provisioning test - utils_test: add test case for mutually exclusive token and tokenFile

- CreateVolume: return InvalidArgument when mountWithOAuthToken is combined with NFS protocol, preventing provisioning shares that will fail at mount time - e2e: add trailing slash to IMDS resource URL (https://storage.azure.com/) to avoid invalid_resource rejection in some environments

NFS can be indicated via either protocol=nfs or fsType=nfs. The mountWithOAuthToken guard only checked protocol, so a PV with fsType: nfs could bypass the check. Reject both.

In CAPZ clusters, the OIDC JWKS document is served from Azure Blob Storage and may not be immediately available after cluster creation. When the CSI driver tries to mount with workload identity, it exchanges the SA token for an Azure OAuth token via AAD. If AAD cannot find the signing key in the OIDC issuer's JWKS metadata, it returns AADSTS7000272 / 400 Bad Request and the mount fails. Changes: - Add 120s wait for FIC and RBAC propagation after setup - Add waitForOIDCJWKS() that polls the issuer's JWKS endpoint until it returns at least one signing key (up to 5 min timeout)

…AuthToken Instead of wrapping the error with codes.Internal, return the error directly so that InvalidArgument and other specific status codes from setCredentialCacheWithOAuthToken are preserved.

…essages - Extract mountWithOAuthToken validation to shared validateMountWithOAuthToken() - Log secret name and namespace when using OAuth token - Improve error message: clientID must be provided when tokenFile is set

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

andyzhangx · 2026-05-17T02:56:51Z

/test

andyzhangx · 2026-05-17T02:58:12Z

/retest

andyzhangx · 2026-05-18T04:11:28Z

/retest

landreasyan · 2026-05-19T00:59:58Z

The secret reference comes from the PV/StorageClass which is provisioned by a cluster admin, but the intention is that the OAuth token represents the user's identity. There's no per-pod scoping and any pod that can mount the PV gets the secret's privileges. Worth a note in the docs that the secret must be treated as cluster-scoped credentials, and secretNamespace should be a namespace not writable by tenants.

andyzhangx · 2026-05-19T01:10:59Z

Security Considerations

added Security Considerations in PR description @landreasyan

landreasyan · 2026-05-19T01:30:45Z

 			setKeyValueInMap(context, secretNamespaceField, context[podNamespaceField])
 			// When Managed Identity is used for ephemeral volumes then reject the request.
 			// Allowing access for inline volume with identity will open up risk of arbitrary pods accessing fileshares with node identity permissions.
 			if strings.EqualFold(getValueInMap(context, mountWithManagedIdentityField), trueValue) {


Could we block the OAuth here for inline volumes explicitly like it is done for the managed identity?

landreasyan · 2026-05-19T01:48:25Z

Since dynamic provisioning is supported, I think we should force the secret namespace to match pod's namespace. Because a user that doesn't have access to view/list secrets in another namespace might be able to exploit the dynamic provisioning path and have csi driver lookup and use the secret with driver's rbac.

Or we can also block the dynamic provisioning path for this auth option.

Explicitly reject OAuth token authentication for inline (ephemeral) volumes, similar to the existing managed identity check. Inline volumes should not use identity-based auth as it would allow arbitrary pods to access fileshares with the provided token's privileges.

andyzhangx · 2026-05-19T02:13:00Z

Since dynamic provisioning is supported, I think we should force the secret namespace to match pod's namespace. Because a user that doesn't have access to view/list secrets in another namespace might be able to exploit the dynamic provisioning path and have csi driver lookup and use the secret with driver's rbac.

Or we can also block the dynamic provisioning path for this auth option.

@landreasyan in dynamic provisioning, we don't do such secret namespace enforcement since storage class is not namespaced, and admin configures storage class. We only force the secret namespace to match pod's namespace in inline volume since pod is namespaced, and pod is managed by normal user. It behaves same as when we use storage account key mount, configure a different secret namespace in CSI driver storage class should be supported.

landreasyan · 2026-05-19T02:17:23Z

Since dynamic provisioning is supported, I think we should force the secret namespace to match pod's namespace. Because a user that doesn't have access to view/list secrets in another namespace might be able to exploit the dynamic provisioning path and have csi driver lookup and use the secret with driver's rbac.
Or we can also block the dynamic provisioning path for this auth option.

@landreasyan in dynamic provisioning, we don't do such secret namespace enforcement since storage class is not namespaced, and admin configures storage class. We only force the secret namespace to match pod's namespace in inline volume since pod is namespaced, and pod is managed by normal user. It behaves same as when we use storage account key mount, configure a different secret namespace in CSI driver storage class should be supported.

We don't need the inline or dynamic support for the issue that was reported. Can we block these scenarios in this PR and merge them later when the security aspects are further discussed? These would be hard to deprecate at a later point.

landreasyan · 2026-05-19T03:10:50Z

Per our discussion the dynamic support seems to be equivalent to the account key authentication path.

/lgtm

andyzhangx · 2026-05-19T03:12:53Z

/retest

andyzhangx · 2026-05-19T04:16:27Z

/retest

andyzhangx · 2026-05-19T05:33:58Z

/retest

andyzhangx · 2026-05-19T07:33:00Z

/cherrypick release-1.35

k8s-infra-cherrypick-robot · 2026-05-19T07:33:41Z

@andyzhangx: new pull request created: #3166

Details

In response to this:

/cherrypick release-1.35

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Apr 21, 2026

k8s-ci-robot requested review from cvvz and gnufied April 21, 2026 11:50

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 21, 2026

andyzhangx force-pushed the mount-with-mi-token branch 2 times, most recently from 552cc56 to c2bd547 Compare April 21, 2026 11:59

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 21, 2026

andyzhangx changed the title ~~feat: support mount Azure File with user-provided managed identity token~~ [WIP] feat: support mount Azure File with user-provided managed identity token Apr 21, 2026

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 21, 2026

andyzhangx force-pushed the mount-with-mi-token branch 2 times, most recently from 4188373 to 18fe595 Compare April 21, 2026 12:15

andyzhangx requested a review from Copilot April 21, 2026 12:18

Copilot started reviewing on behalf of andyzhangx April 21, 2026 12:18 View session

andyzhangx force-pushed the mount-with-mi-token branch from 18fe595 to fa39190 Compare April 21, 2026 12:20

andyzhangx changed the title ~~[WIP] feat: support mount Azure File with user-provided managed identity token~~ feat: support mount Azure File with user-provided OAuth token Apr 21, 2026

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 21, 2026

Copilot AI reviewed Apr 21, 2026

View reviewed changes

Comment thread pkg/azurefile/utils.go

Comment thread pkg/azurefile/nodeserver.go

Comment thread pkg/azurefile/azurefile.go

Comment thread pkg/azurefile/nodeserver.go Outdated

Comment thread pkg/azurefile/nodeserver.go Outdated

Comment thread pkg/azurefile/utils_test.go

andyzhangx changed the title ~~feat: support mount Azure File with user-provided OAuth token~~ [WIP] feat: support mount Azure File with user-provided OAuth token Apr 21, 2026

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 21, 2026

andyzhangx force-pushed the mount-with-mi-token branch 2 times, most recently from 6f727c1 to 89e66ed Compare April 21, 2026 13:27

andyzhangx requested a review from Copilot April 21, 2026 13:33

Copilot started reviewing on behalf of andyzhangx April 21, 2026 13:33 View session

Copilot AI reviewed Apr 21, 2026

View reviewed changes

Comment thread pkg/azurefile/nodeserver.go Outdated

Comment thread pkg/azurefile/azurefile.go Outdated

Comment thread pkg/azurefile/utils_test.go

andyzhangx force-pushed the mount-with-mi-token branch 2 times, most recently from 5c07d5e to 927abaf Compare April 21, 2026 14:01

andyzhangx requested a review from Copilot April 21, 2026 14:03

andyzhangx added 9 commits May 16, 2026 14:26

fix: return codes.Internal for setCredentialCache failures

11fd442

Use status.Errorf instead of fmt.Errorf so the error surfaces as codes.Internal rather than codes.Unknown through status.Convert.

fix: return codes.Internal for secret fetch failures

4f17e7b

Use status.Errorf instead of fmt.Errorf so the error surfaces as codes.Internal rather than codes.Unknown through status.Convert.

fix: also reject mountWithOAuthToken when fsType is nfs

cc88f7d

NFS can be indicated via either protocol=nfs or fsType=nfs. The mountWithOAuthToken guard only checked protocol, so a PV with fsType: nfs could bypass the check. Reject both.

fix: propagate original gRPC status code from setCredentialCacheWithO…

ac1d9d2

…AuthToken Instead of wrapping the error with codes.Internal, return the error directly so that InvalidArgument and other specific status codes from setCredentialCacheWithOAuthToken are preserved.

fix: remove stale comment above validateMountWithOAuthToken

44d2691

Copilot AI reviewed May 16, 2026

View reviewed changes

Comment thread test/e2e/suite_test.go Outdated

Comment thread test/e2e/suite_test.go Outdated

Comment thread go.mod

andyzhangx added 3 commits May 16, 2026 14:34

fix: remove dead code and duplicate line from rebase

46bff09

fix: move azidentity from indirect to direct dependency in go.mod

7a4d552

test: increase storage account limit from 17 to 20 in AfterSuite check

2e7cbc1

landreasyan reviewed May 19, 2026

View reviewed changes

k8s-infra-cherrypick-robot mentioned this pull request May 19, 2026

[release-1.35] feat: support mount SMB Azure File with user-provided OAuth token through k8s secret #3166

Merged

Conversation

andyzhangx commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this?

What this PR does / why we need it:

Changes:

Usage Example

1. Create Secret with OAuth Token

2. Dynamic Provisioning (StorageClass)

3. PV (Static Provisioning)

Security Considerations

Which issue(s) this PR fixes:

Does this PR introduce a user-facing change?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andyzhangx commented May 17, 2026

Uh oh!

andyzhangx commented May 17, 2026

Uh oh!

andyzhangx commented May 18, 2026

Uh oh!

landreasyan commented May 19, 2026

Uh oh!

andyzhangx commented May 19, 2026

Uh oh!

landreasyan May 19, 2026

Choose a reason for hiding this comment

Uh oh!

andyzhangx May 19, 2026

Choose a reason for hiding this comment

Uh oh!

landreasyan commented May 19, 2026

Uh oh!

andyzhangx commented May 19, 2026

Uh oh!

landreasyan commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

landreasyan commented May 19, 2026

Uh oh!

andyzhangx commented May 19, 2026

Uh oh!

andyzhangx commented May 19, 2026

Uh oh!

andyzhangx commented May 19, 2026

Uh oh!

andyzhangx commented May 19, 2026

Uh oh!

k8s-infra-cherrypick-robot commented May 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

andyzhangx commented Apr 21, 2026 •

edited

Loading

landreasyan commented May 19, 2026 •

edited

Loading